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INTRODUCTION 

One of the most difficult problems in the administration 
of a rehabilitation program for mental patients centers 
around the proper selection of patients for training. Usually 
there are many more patients than spaces available and the 
problem is to avoid slowing the rate of turnover by selecting 
patients who can be expected to 'graduate* more rapidly. The 
converse of this problem is the tendency of staff to overlook 
the quiet, unobtrusive patient who does little to attract 
attention. 

What is needed is a simple way to organize or order a 
group of patients in terms of their probable success in 

t 

rehabilitation training. Preferably this method should makQ 
minimal demands on professional staff time, require only 
simple equipment, and be administered and scored by aides 
with a minimum of additional training. 

We believe that the system to be described in this report 
will fulfill most of these requirements, provided that it is 
used with care and interpreted with due regard to its stated 
limitations. The method is clearly an adjunct to and not a 
substitute fox* clinical judgment. Por example, the tests can 
indicate that a paranoid patient is capable of psychomotor 
function which would enable him to hold employment outside 
the hospital, but it will not reflect the fact that his 
delusional system may make his release inadvisable and employ- 
ment impossible. Such evaluation is currently, and may well 
remain, in the province of clinical judgment. 

This report will be concerned with the use of the 
battery of psychomotor tests in selecting for rehabilitation 
training mental patients of varying diagnoses from the 
population of a state mental hospital. 

The use of psychomotor performance measures on mental 
patients is not new. There has been a persistent effort 
over nearly the last one hundred years to apply these 
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instruments to varying population of mental patients. Most 
efforts were made either in the hope of developing a differ- 
entially diagnostic tool or in distinguishing the ways in 
which the mental patient differed from the normal. 

The first aim appears to have been completely frustrated. 
To our knowledge, there have been no successful attempts to 
categorize mental patients into diagnostic groups on the 
basis of test performance. Considering the imprecision of 
diagnostic criteria, the failure may not be ascribed entirely 

to the psychomotor measures. 

Attempts to differentiate the normal from the mentally 
ill have likewise been unsuccessful so long as the individual 
rather than the actuarial approach has been used. It has 
been an almost universal finding that the mentally ill perform 
more poorly than do normals and that the degree of impairment 
is correlated with the severity of the mental illness. 

However, there is sufficient overlap in the performance 
ranges for the two groups to make *he assignment of a given 
individual to the normal or mentally ill group so imprecise 
as to be clinically useless. 

This report will not be concerned with an elaborate 
review of the literature concerning psychomotor tests and 
mental patients. The interested reader is referred to the 
list of references for a selection of titles which will indi- 
cate the diversity to be found in the area. An excellent 
review of the literature is to be found in King (1954). Aft- 
exhaustive bibliography of titles dealing with psychomotor 
performance, not limited to mental patients, has been pub- 
lished by Ammons and Ammons (1964). In recent years there 
has been an increased use of psychomotor test performance in 
the analysis of drug effects. (Benjamin et al 1957* Lehman 
and Csank 1957, Karnetsky and Humphries 1958, DiMascio and 
Henkel I960, Peehkin 1962, Klugman 1962, Pearl 1962) 
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Broadly summarized, their findings might be said to 
indicate that tranquilizing medication tends to adversely 
affect normal subject performance and to improve that of 
mental patients, a finding in harmony with the numerous 
reports of impaired performance in mental patients. 

The work done in our laboratory on psychomotor testing 
of mental patients has taken several directions. Initially 
we partially confirmed the findings of king (1954) with 
respect to the differentiation of degree of mental illness 
in terms of psychomotor test performance (Weaver 1961). 

The effects of drug-induced Parkinsonism on psychomotor 
performance of schizophrenic patients was investigated 
(Weaver and Brooks 1961) with the finding that the psycho- 
motor test battery was differentially sensitive to Parkinsonism. 
Pine coordinated movements were affected first and to the 
greatest degree. 

A study of the relationships between psychiatric status 
and psychomotor performance (Brooks and Weaver 1961) indicated 
clearly that psychomotor test performance detected the 
exacerbation of psychotic symptoms under placebo medication 
before this became clinically apparent and also showed 
evidence of recovery under tranquilizing medications before 
the clinical manifestations of psychosis were reduced. 

These findings led to a study of the use of psychomotor 
test performance as a means of predicting rehabilitation 
readiness or potential of schizophrenic patients (We aver and 
Brooks 1963). This study indicated that the psychomotor test 
battery could select, at a statistically significant level, 
those patients who would eventually leave the hospital. 

Cutting points could ba selected ttfhich would maximize the 
selection of successful patients or minimize the selection of 
unsuccessful patients, according to the purposes for which 
the test was being given. It should be emphasized that this 
was a concurrent study of a rehabilitation process already 
in operation. The tests were not used to evaluate patients 



o 



for rehabilitation training or release, thus maintaining the 
necessary independence between criterion and test measures. 

The approach assumes that substantially all the patients ready 
for discharge will be detected and released, an assumption 
made reasonably tenable by the very active rehabilitation 
program at the research hospital. 

The success of this technique applied to schizophrenic 
patients raised the question of application to the general 
population of a state mental hospital. The remainder of this 
report will be devoted to this topic. 

For the benefit of readers interested in the research 
milieu, a brief description of the research hospital follows. 

The Vermont State Hospital, the only public mental institution 
in the state, is a medium— sized hospital with an average 
patient population of 1220 patients. There are about 700 
admissions or readmissions and 575 discharges per year. 

There has been a slight reduction in the average daily popu- 
lation in the last ten years but a marked increase in the 
number of admissions. The hospital admits all types of 
severe mental disorders, except for the mentally retarded, 
from the entire state of Vermont. The hospital currently em- 
ploys 11 physicians, 20 nurses, 4 social workers, 3 clinical 
psychologists, 2 teachers, a recreational therapist, an 
industrial therapist, and 275 psychiatric aides. It has an 
active research department, operated in conjunction with the 
University of Vermont, which employs a full-time sociologist, 
an experimental psychologist, 3 research assistants, 4 
secretaries, an electronics technician, and 4 vocational 
instructors. 

At Vermont State Hospital there is a strong rehabili- 
tation program. One aspect of this is to provide basic 
vocational training for small groups of chronic patients 
geared to agricultural, industrial, domestic, nursing, 
maintenance, and commercial skills. The aim of the in-hospital 
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vocational training program is to prepare patients to leave 
the hospital via the use of a rehabilitation house. 

It is tied to the rehabilitation house program which 
is sponsored by the Vermont Department of Vocational 
Rehabilitation. The principle of the ' half~way houses' is 
to offer residence for a period of months. During this time, 
through the assistance of a vocational counselor, the 
residents work in the community and become accustomed to 
community life on a graduated basis with the result being a 
full return to community living. 



o 

ERLC 



2.1 



Chapter 2 
METHODOLOGY 

The basic design of this study is the classic industrial 
psychology approach to test validation. In this design the 
instrument under study is administered to a selected sample 
of the population. The results are not allowed to influence 
the fate of the sample members* When a sufficient time has 
elapsed to make possible a reasonably stable estimate of the 
success or failure of the subjects, the test results are com- 
pared with the outcome data to determine whether the tesu 
results could be used to select individuals more efficiently 

than the method currently in use. 

The method has several limitations which must be con- 
sidered in connection with the present; study • Primarily , it 
assumes complete independence of the test and criterion 
measures in the sense that test performance is not allowed 
to affect either the subsequent actions of the subject nor 
evaluation of subject performance made by a third party. 
Failure to control this aspect results in a form of circular 

validity. 

The second limitation is that the method assumes a 
virtually unrestricted sample of the population to which it 
is to be applied. This is necessary to prevent discard of 
sample members who are eliminated for other reasons. This 
limitation does not operate to affect the definition of the 
population to ifhich it is to be applied but does require 
unrestricted sampling of the defined population. 

A third limitation is that a sufficient amount of time 
must be allowed for post-test criterion performance. Lack 
of sufficient time tends to decrease the sensitivity of the 
criterion measure and obviously, the validity of a test is 
limited by the criterion measures. 
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Turning now to the design of the current study, the 
research question concerned the ability of the psychomotor 
test battery to determine which patients from the general 
hospital population would be released. The general hospital 
population is defined as all patients with certain exclusions 
made for largely pragmatic reasons. The first excluded 
category is patients confined through legal rather than 
medical process, an exclusion justified by the fact that the 
criterion measure (leaving the hospital during the post-test 
observation period) is influenced by the legal aspects and 
differs markedly from the criteria applied to the ordinary 

patient. 

The second exclusion is based on chronological age. 

As the only state mental institution, a large proportion of 
the resident population consists of elderly patients who are 
management problems for their families. There is little 
point in including these essentially terminal patients in a 
population of potential rehabilitation candidates since many 
would be unemployable at best. Hence a cut-off age of 60 
years was used; a figure also used by the hospital in deter- 
mining eligibility for rehabilitation training. In practice, 
a cut-off date of age 58 date of testing was employed in 
order to assure compliance with the criterion used by the 
hospital. In view of the limited facilities for rehabilitation 
training available in most mental hospitals, the age limit 
is probably over— inclusive at the upper end. 

At the lower end of the age scale, we have excluded 
only children not yet in the teens. The reason is simply 
that it is the lower adult age brackets which are mosf.likely 
to receive rehabilitation training and hence the group for 
which a predictive device would be most useful. 

The third limitation mentioned — a sufficient length 
of post-test experience — was likewise a matter of pragmatic 
balancing-off of various factors. Experience tables at the 
research hospital indicates the median length of stay at 
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discharge for all patients is 2.2 months and about 3-2 months 
for psychotics. The mean length of stay for all patients was 
6.7 months and about 10.8 months for psychotics. When con- 
sideration is given to the fact that the mean can be unduly 
influenced by the addition of a few long-stay cases, it seems 
clear that a post-test observation period of one year would 
include most of the cases who could be considered currently 
to possess much potential for rehabilitation. The purpose of 
the test battery was to select those patients who would profit 
most from rehabilitation training in terms of being ready to 
benefit from it. We felt that hospitalization for one year 
would indicate that the patient was, at the time of testing, 
less ready to benefit from training than the patient who was 

discharged within the year. 

We are aware of the limitations and assumptions involved 
■in this system, most of which are inherent and common to any 
research Involving the use of human subjects in a therapeutic 
situation. The crucial assumption is that all patients who 
sh0U ld be released in fact ar® released. . This assumption is 
partially justified on the grounds that the hospital has a 
strong tradition of non-custodial treatment as well as an 
active rehabilitation effort. Evaluative procedures designed 
to prevent overlooking of recovered cases are systematically 
carried out. While the system is far from perfect in view of 
the small staff -patient ratio, it does seem to minimize a 
potentially serious source of error. 



Subjects 

The population for this study was defined as all patients 
in the Vermont State Hospital who were residents or who were 
admitted during the course of the study. As shown m Table 
2.1, a total of 2175 cases were available. A total of 1203 
useable records were obtained from this population. The 
majority of the losses were incurred in the 'over-age 1 and 
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Expired’ categories. The over-age group included all 
patients aged 59 years or older at the time of testing. 
Hospital policy was not to select for rehabilitation train- 
ing any person over 60 years of age. Since this could 
affect the discharge rate independently of the factors 
under study, the older group was not tested. Inspection 
of Table 2.1 shows a disproportionately heavy concentration 
of chronic brain syndrome patients in this group, probably 
indicating a large number of senile arteriosclerotic s. 

Hence the exclusion seemed warranted. 

A rather striking fact was that the incidence of un- 
testable patients remained fairly low. Only 112 of 1376 
patients (8%) could not be induced to cooperate. The fact 
that the tests could be applied to 90% of a mental hospital 
population selected only on the basis of chronological age 
may be accepted as evidence of the wide applicability of 
the technique. 

A basic question is to what extent our group is truly 
representative of the population from which it was drawn. 
Obviously, there were non-random factors (age and testability) 
at work in the selection process. 

At the bottom of Table 2.1 is presented the per cent of 
the total group falling in each diagnostic category. As might 
be expected, the majority of the cases fall in the schizo- 
phrenic and chronic brain syndrome categories. For ease in 
comparison, the same analysis is presented on the next line 
for the tested group. Only two major discrepancies were 
observed; an increase in the schizophrenic and a decrease in 
the chronic brain syndrome categories. The reason for these 
changes lies in the uneven distribution of the over-age group* 
Only 21% of the schizophrenic group was over-age while 49% 
of the chronic brain syndrome group fell in that class. The 
incidence of untestable was similar in both groups. 

It was concluded that, for the purposes of this paper, 
the sample of 1203 cases does not differ significantly from 
the patient population. 
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One further correction was applied to the sample which 
reduced the size even more. While no control was attempted 
over such factors as number or length of previous hospitali- 
zation, it was deemed necessary to establish a minimum period 
of observation following testing. A common criterion has 
been outcome at one year following admissions. In this case 
we adopted as the criterion " status at the end of one year, 
following testing*’ with certain modifications. Patients dis- 
charged during the year were classified as 'out*. Patients 
expiring with less than one year experience were dropped. 

Table 2.2 gives the changes in the group occasioned by the 

application of the one— year criterion. 

It is obvious that the reduction did not materially 
alter the inter-group distribution. Considered in terms of 
per cent of the total, no diagnostic group changed more than 
one per cent. This group, as constituted, will be used in 
the investigation of the question of whether the different 
diagnostic groups perform differently on the psychomotor tests 
The factor of age is not particularly germane to the 
purpose of this study except that cognizance must be taken 
of the possible introduction of bias if the diagnostic groups 
should vary widely in age. The distribution of the 1184 
patients by age and diagnosis is presented in Table 2.3 in- 
terns of number of cases and in per cent of each diagnostic 

group. 

Test Battery 

The apparatus used in this study was developed in our 
laboratory during the first part of our research program. 

It has been reported elsewhere (Weaver and Brooks 1961) • 

For this study, the test battery consisted of five tests. 

Reaction time. In this test the subject (S) was seated in 
a chair facing the apparatus, with the right forefinger 
resting on a telegraph key. At the warning from the 
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experimenter (E) , S depressed the key and attended to the 
stimulus lamp. At the stimulus onset, the light plus a 
tone of medium intensity, S lifted his forefinger from the 
depressed key. The elapsed time was measured by a .01 second 

electric timer. 

Hate of Tapping. This test was designed to measure the 
ability of the patient to perform a rapid repetitive 
movement. S was seated in a chair facing the telegraph key. 
Eollowing instructions to tap as rapidly as possible, o was 
given the verbal signal to begin. When S had begun to tap, 

E turned on the recording counter for five seconds. This 
methow gave a sample of an on-going response unaffected by 
response time. 

T ransport-Assembly Test. This test is basically a modifica- 
tion of the Purdue Grooved Pegboard as described originally 
by Tiffin (1952). The test consisted of the pegboard, a 
magazine for the pegs, and the necessary circuitry for a 
capacitance-operated relay system. 

The pegboard was about eight inches square and was 
drilled with a symmetrical pattern of 25 holes. Over this 
was a cover with a matching pattern of holes. Each hole in 
the cover has a slot or keyway extending radially from the 
edge of the hole. Each peg had a short stud projecting 
radially from its middle. The task of the subject was to 
procure a peg from the magazine, transport it to the board, 
insert it into a hole while aligning the stud with the slot, 
depress it against light spring pressure, and rotate the stud 
underneath the cover to retain the peg in place. 

The task can be conveniently divided into two elements; 
transport of the hand to and from the magazine, and manip- 
ulation which consists of grasping the peg at the magazine 
and later inserting and twisting it in the hole. The times 
for these elements were cumulated separately and were recorded 
as 'Transport' and 'Assembly' scores respectively. Each were 
considered as a separate test in the analysis of the data. . 
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S eria l Beaction lime. This test consisted of five brass 
target plates, 1 1/2 inches in diameter, linerally arrayed 
on a horizontal surface 8 inches by 24 inches. The plaues 
were spaced with 2 1/2 inches between adjacent edges. Each 
target had an associated red stimulus light. The task of the 
standing S was to touch with a wand the plate whose stimulus 
light was currently glowing. This response extinguished that 
lamp and turned on the next in the series. Twenty such re- 
sponses constituted one trial. The order in which the lamp^ 
came on could be varied among five patterns. The patterns 
were so arranged that no stimulus light succeeded itself. 

The score consisted of the time necessary to complete the 
twenty responses. Five trials, one on each pattern, were 

given at each session. 



At the initial testing session, the patient was given a 
verbal orientation and aemonsuration ox oho pexio^m^nc^ re 
quired. At least one practice trial was allowed before test- 
ing began. The amount and type of orientation allowed varied 
with the needs of the patient. The obj active was to get the 
best estimate of the level of performance of which the patient 
was currently capable. On succeeding days, less orientation 
usually was required. In no case was data taken until the E 
was confident that the S understood and would cooperate. 



In this connection, it was found best to encourage the 
patient to do his best work but also it was important to 
avoid any suggestion of pressure or competition. Best per- 
formance seemed to occur when the patient was alert but 
relaxed. Although both speed and precision were emphasized 
in the measures, this emphasis was not necessarily reflected 
in the performance of the patient. It has been our experience 
that when there is a conflict between speed and precision, 
the patients usually sacrificed speed in favor of precision. 
Hence the performance as measured here probably represents 
the more usual level of function of the patient rather than 
the absolute maximum performance of which he was capable. 



lor those interested in the apparatus, a complete wiring 
diagram and perspective drawings will be found in the appendix 
to this report. 

Table 2.4 gives the essential data collection plan. 

The data collection session required about ten minutes for 
an experienced subject and about twice that for inexperienced 
or deteriorated subjects. There was little evidence that 
the S’ s were bored or generally slack, but it would be idle 
to assume that each S was performing at maximum capability 
during each session. It seems probable that the data repre- 
sented a better approximation of the ’normal 1 operating leve3. 
for the patient. While the S was encouraged and urged to do 
his best, it was found necessary to keep the ’pressure' down 
to avoid excessive disturbance of the subjects. 

The data reported here were from the initial test ex- 
perience, obtained either during the hospital-wide testing 
of resident patients or as soon as possible after the 
admission diagnosis. 



Table 2.4 



Data collection plan for the study. 



Test 


Unit 


Duration o'r Number 
of Samples per Day 


Reaction Time 


.01 sec. 


5 samples 


Tapping 


each 


2 five-second samples 


Serial Reaction Time 


.01 sec. 


Time to complete 20 
responses; 5 samples, 
1 on each sequence 


Transpoi't -Assembly 


.01 sec. 


Time to place 25 pegs, 
1 sample 
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Chapter 3 

DIAGNOSIS RELATED TO PSYCHOMOTOR PERFORMANCE 

Before statistical treatment of the data was undertaken, 
an analysis was made of the relationship between the criterion 
measure, which was released or not released from the hospital 
within one year after testing (abbreviated hereafter as 
In/Out), and diagnostic category. The purpose of this 
analysis was to determine whether the diagnostic categories 
differed sufficiently in distribution on the criterion 
variable to cloud interpretation of statistical analysis* 

Prom Table 3.1 it is evident that there is some rela- 
tionship between diagnosis and outcome. Even allowing for 
the relatively small numbers in certain diagnostic categories, 
it seems quite evident that a fortuitous combination of 
circumstances could heavily weight the data. The alcoholic 
and psychoneurotic categories show a very high rate of 
discharge. If they also performed well (or even poorly) as 
a group on the psychomotor battery, the interpretation of 
the results on the other categories could be clouded. 

The next step was to determine the relationship between 
diagnosis and psychomotor performance at a semi-quantitative 
level in order to check for this source of error. The 
determination was made for each test separately by arranging 
the scores in order of merit, dividing them into deciles of 
about 120 scores each and then categorizing them by diagnosis 
at each decile level. The results of this analysis combined 
across the five tests are presented in Table 3.2. 

Prom this table it is immediately apparent that there 
is inequality of distribution. In fact only the two psychotic 
categories, schizophrenic and manic-depressive, approach an 
even distribution above and below the group median (upper 
limit of decile V). All other categories show about an 8? 2 
split at this point. 
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The decision was made to discard from further analysis 
the psychoneurotic and alcoholic categories for the following 

reasons: 

(1) There was a marked interaction between 
performance and outcome for both groups; 
both had more than 80% above the median 
and more than 95% out. Inclusion might 
well give rise to spurious accuracy when 
considering the diagnostic groups as a whole. 

(2) There is little need for selecting from 
these groups for rehabilitation training 
since so high a percentage leaves the 
hospital without such "training. 

The same line of argument may be as legitimately applied 
to the mentally deficient category and, with lesser force, to 
the chronic brain syndrome and personality disorder groups. 

It was, however, decided to retain these groups primarily 
because they form a considerable fraction of the hospital 
population from which rehabilitation training candidates must 
be selected. While the chronic brain syndrome and mentally 
deficient groups have a relatively low rate of release, there 
is a continuing movement out of the hospital, and it is 
possible that more efficient selection procedures could 
increase this rate. 

We may now proceed to a formal examination of the relation 
ship between diagnostic category and psychomotor test per- 
formance. In accord with the previous decision the alcoholic 
and psychoneurotic groups were dropped from consideration. 

We also dropped all cases with incomplete score records. Thus 
the five diagnostic categories and the number of cases in each 

were: 



Group No. 


Diagnosis 


N 


1 


Schizophrenic 


636 


2 


Manic Depressive 


84 


3 


Chronic Brain Syndrome 


102 


4 


Personality Disorder 


91 


5 


Mentally Deficient 


83 



3.5 



Our test program was designed to yield five daily scores 
for each, individual on each test. These five scores permitted 
us to use a repeated-measures analysis of variance model which, 
by fractioning the error term permitted a more sensitive 
P-test. The general statement of the model and its degrees 



of freedom appears below. 

Source of Variance df 

Between Individuals 995 

Groups 4 

Subjects within Groups 991 

Within Individuals 5984 

Days 4 

Days x Groups 16 

Days x Subjects within Groups 5964 



The results of this analysis of variance are briefly 
summarized in Table 3«3« The complete results are contained 
in the appendix. 

Table 3-3 

Summary of the analysis of variance results for the 
groups and days analysis. Cell entries are level 
of significance of the P-ratios for the 
combinations of tests and variance sources. 

Tests 



Source of Variance 


Tapping 


Reaction 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

Reaction 

Time 


Groups 


.01 


.01 


.01 


.01 


.01 


Days 


.01 


.01 


.01 


.01 


.01 


Days x Groups 


.01 


NS 


.01 


.05 


.01 
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Our initial attention must be given to the fact that 
four of the five interaction terms have a significant F-ratio. 
This indicates that certain combinations of days and groups 
interacted in a manner not wholly explicable from the days 
and groups main effects. Hence it is possible that the main 
effects may be significant largely due to this interaction. 

In statistical terms, a significant interaction clouds the 
interpretation of the main effects. 

Investigation of this interaction was made by constructing 
figures showing performance as a function of days for all 
groups on each test. These figures are reproduced in Appendix 
I. It was quite evident that the interactions were due in 
the main to minor variations in the day-to-day performance. 

To summarize briefly: 

(1) In no case did any curves cross, though 
in two instances they were virtually 
congruent. 

(2) With only one exception, the groups 
maintained their hierarchial position 
from test to test. 

(3) The vertical separation of the groups 
and the virtually universal (one obvious 
exception) improvement in performance 

as a function of daily testing make it 
abundantly clear that two significant 
main effects (days and groups) are in 
fact real and are not an artifact due 
to interactions. 

Next we may turn to the analysis of the two main effects, 
groups and days. The effects for days is, of course, strictly 
a learning phenomenon, and we will dismiss it from further 
consideration on the grounds that the results conform to 
those of previous studies. It should be noted, however, that 
the slope of these curves varies with the different diagnostic 
groups, and it would be quite misleading to pool them into a 
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single function. There is no learning curve for 'mental 
patients', a fact commented on elsewhere h y King (1954). 

The main effect for groups was significant at the .01 
level for all five tests. This statement gives no indication 
of the magnitude of these differences. The observed F-ratios 
varied between 40 and 117 where a ratio of 5*35 is significant 
at the .01 level. In short., they were really different. 

This brings up the question of who differs from whom and 
in what respect. An intuitive answer can be gained by in- 
spection of the figures of Appendix I which shows performance 
as a function of days for each diagnostic group. There is 
really a wide range of performance among the groups, but the 
question of who differs from whom requires a statistical 
decision-making process. The appropriate technique for this 
analysis is the Newman-Keuls test of the significance of 
differences among ordered means. The results are presented 
in Table 3.4 which shows the significance level of the 
differences among the group means on all tests. 

Thab table may be conveniently summarized by considering 
the presence or absence of significance and rearranging the 
sequence of groups into the order-of-merit sequence. The 

results are presented in Table 3*5* 

From these tables it seems obvious that there were real 
and sizeable differences in performance among the diagnostic 
groups. There are ten comparisons within each test and five 
tests for a total of fifty comparisons. Of these, 31 were 
significantly different. The mentally deficient entered into 
16 such comparisons , the chronic brain syndrome into 13 ? 
personality disorder into 12, schizophrenic into 11 * and 
manic depressive into 10. (These figures sum to 2N — 62 
since each comparison is counted once for each group.) 

Clearly, the mentally deficient group differed most often 
from all other groups. If we eliminate this group from further 
consideration for the moment, the maximum number of significant 
comparisons possible now becomes 6x5 tests » 30. Of these, 
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13 we re actually significant. The chronic brain syndrome 
group participated in 12 of these, personality disorder in 
7, schizophrenic in 6, and manic depressive in 5* It seems 
fair to say that the mentally deficient and chronic brain 
syndrome groups accounted for the major share of the sig- 
nificant differences observed, yet only once was there a 
significant difference observed in five comparisons between 
these groups. Similarly the other three groups differed 
among themselves only three times in fifteen comparisons. 

Of course, it is not particularly surprising to find 
that the chronic brain syndrome and mentally deficient 
patients should tend to differ from the other diagnoses more 
often than from each other, but it is much less expected 
that diagnostic groups as far apart in terns of degree of 
pathology as schizophrenic and personality disorder should 
not differ more widely than on two of five comparisons. 

In summary, analysis of the significance of the dif- 
ference among group means seems to indicate that the majority 
of the differences occurs between two groupings which may 
be loosely termed 1 functional ' (schizophrenic, manic de- 
pressive, and personality disorder) and ’organic' (mentally 
deficient and chronic brain syndrome). The matter of 
differential performance among the diagnostic categories 
will be considered again when the problem of cutting scores 
is discussed. 
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Chapter 4 

PSYCKOMOTOR TESTS RELATED TO PATIENT OUTCOME: 

MULTIPLE REGRESSION AND MULTIPLE CORRELATION METHOD 

This chapter concerns the ability of the tests to dif- 
ferentiate those patients who will leave the hospital from 
those who will not leave within the one-year observation 
period. Demonstration that there is such a differential per- 
formance must logically precede attempts to set up selection 

procedures. 

The appropriate analysis of variance model for this 
problem is the p x q with repeated measures on the second 
factor. The model is identical to the one used in the pre- 
ceding analysis except that the A factor now has two levels 
(in or out of hospital) instead of the five levels (diagnostic 
groups) as in the first analysis. The advantage of retaining 
this approach is that it divides the residual into two parts, 
thus increasing the sensitivity of the P-test. The model is 
shown in Table 4.1. 

As may be seen from the table, there are three P-ratios 
for each group-test combination to be evaluated. The level 
of significance for each is reported in Table 4.2. 

Of the results appearing in Table 4.2 only certain 
aspects will merit close attention. The Days main effect 
and the In/Out x Days interaction are essentially what has 
been seen in the previous analysis. The Days effect, sig- 
nificant in 22 of 25 instances, merely indicates learning 
during the course of the test administration, and the sig- 
nificant interaction term indicates that learning was not a 
constant for all days. It is noteworthy that there was no 
significant interaction in the chronic brain syndrome, 
mentally deficient, or personality disorder groups. 

We are primarily interested in the In/Out main efxect. 
Overall, this effect was significant in 20 of 25 instances 
with all five of the non-significant instances occurring in 
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Model for analysis of variance used in investigation 
of relationship to outcome to psychomotor performance. 
Degrees of freedom vary with size of groups. 



Source_of Variation Degrees of Freedom 

Between Subjects 

Outcome 1 

Subjects within Groups (n-1 ins) + (n-1 outs) 

Within Subjects 

Days 4 

4 
4 

(n in + n out - 2) 



Days x Outcome 

Days and Subjects 
within Groups 
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the mentally deficient group. Only two of these twenty in- 
stances were significant at the lox^er .05 level. Both of 
these were in the chronic "brain syndrome group, thus con- 
tinuing the association of these groups previously found in 
the between-diagnostic-groups analysis reported in Chapter 3* 

A review of the 18 F-ratios involved showed them to be quite 
substantial, ranging from 8.24 to 167.14 where F = 6.70 is 
significant at the .01 level. We may conclude, therefore, 
that there exists a sufficiently substantial difference in 
performance between the ’in 1 and ’out' groups to warrant 
further investigation of the tests as a selection device. 

There are a variety of approaches possible in the 
analysis of the tests as a selection device. We shall be 
concerned with two; the use of multiple cutting points and 
the analysis of regression. The first of these is a rather 
pragmatic cut-and-fit procedure, while the regression analysis 
is statistically more elegant, although there is no guarantee 
that the product will be superior. 

The multiple regression and multiple correlation method 
of analysis is one of the more sophisticated statistical 
techniques applicable to the problem of determining the weights 
to be accorded to the various tests in a test battery. For 
those readers who are not acquainted with the method, we will 
attempt to develop a non-technical explanation which itfill 
give the reader an intuitive grasp of how the technique works. 

The basic elements in this situation are the age and sex 
factors, the five psychomotor test scores, and the outcome for 
each individual. We will refer to the factors and the tests 
as predictor variables, and to the outcome as the criterion 
variable. The relationship between each predictor variable 
and the criterion variable is expressed by the coefficient of 
correlation which may have any value from -1.0 through aero to 
+1.0. Using these coefficients as input data and using the 
analysis of regression statistical technique, we can select a 
weighted combination of tests which will give us the best 
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prediction of our criterion measure. The statistic called 
multiple correlation is now computed which measures the 
degree of relationship between our predicted results and what 
was actually observed. 

To summarize briefly, we measure the relationship between 
each of our psychomotor tests and patient outcome, use these 
measures to select the best weighted combination of tests , 
use this combination to make predictions , and then measure the 
relationship between predicted and observed outcomes. 

In this analysis we will use an approach developed in 
industrial psychology for evaluating selection procedures 
(Tiffin 1952). The method requires a large group of subjects 
for whom test scores and outcome data are available. The 
group is divided into a primary and a criterion group on some 
basis other than test scores or outcome. The selection pro- 
cedures are then developed on the data for the primary group, 
endeavoring to select those measures which will best predict 
outcome. The combination of tests thus selected is now applied 
to the criterion group data and evaluated in terms of how well 
outcome was predicted. 

The schizophrenic group was selected because it was the 
largest- -of the diagnostic groups and had the best balance be- 
tween the ’in’ and 'out' outcome classification. To form 
the 'primary* and 'criterion' sub-groups, the 636 cases were 
arrayed in order of age at testing and divided alternately 
into two groups of 3i& cases each. Each sub-group was further 
divided into an 'in' and 'out' group on the basis of actual 
outcome. The characteristics of the groups thus formed are 
given in Table 4.3» 

In this analysis we will use the schizophrenic primary 
group to obtain the test weights and then apply the weights 
to the data for the schizophrenic criterion group and see how 
well the predictions fit the actual outcome. Next we will 
apply them to the other diagnostic groups to see how well they 
work here. Finally we will pool all 996 cases, obtain 
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Table 4.5 

The distributions of the primary and criterion 
groups with respect to outcome. Cell entries 
are the number of cases. 



Group 


Out 


In 


N 


Primary 


177 


141 


518 


Criterion 


186 


132 


518 


Totals 


365 


273 


656 



regression weights based on the entire group, and examine 
accuracy of prediction among the various diagnoses. 

In order to do this analysis, it will be remembered that 
we need the various correlations among the test scores and of 
the predictors with the criterion variable. The inter- 
correlation matrices are presented in the statistical 
supplement. Table 4.4 shows the correlation between pre- 
dictor and criterion variables. Caution must be observed 
in interpreting the signs. In all of the psychomotor tests 
except rate of tapping, a low score is a good score. The 
code for ‘in* and ’out* has the numerical value for 'out* 
greater than that for f in'. Hence a negative correlation 
would be expected if goodness of performance is related to 
discharge. The negative sign for the age factor indicates 
that younger patients tend to get out more readily, an 
observation not at variance with common experience. The only 
exception to this rule is seen in the chronic brain syndrome 
group which has a positive correlation between age and 
discharge. Investigations indicated that this was due to 
minor fluctuations in small numbers. If we categorize age 
in five year inte3?vals, the difference in number getting out 
between the intervals with the highest and lowest frequency 
is three patients. 
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All correlations are significantly different from zero 
at the .01 level except those marked with an asterisk which 
are significant at the .05 level and those underlined which 
are not significant. 

It is instructive to note that the mentally deficient 
group differs markedly from the other diagnostic categories. 

It has the lowest correlations of any of the groups on the 
psychomotor tests. The difference is probably due to the marked 
imbalance between the numbers in the 'in’ and ’out' groups. 

Only 8 of 83 patients in this group got out of the hospital 
during the year after test. 

The following observations can be supported from obser- 
vation of Table 4.4. 

(1) The diagnostic groups characterized by 
brain dysfunction (chronic brain syndrome 
and mentally deficient) tend to have 
lower correlations of test scores with 
the criterion. 

(2) The correlations involving age and sex 
tend to be lower than any of the other 
factors. 

(3) Of the 14 correlations not significantly 
different from zero, 6 occur in one 
diagnostic group and the other 8 are 
found in the age and sex factors. 

(4) Of the 35 correlations which are signifi- 
cantly different from zero, none account 
for the large share of the variance. 

It must be remembered, however, that 
the predictor variable is dichotomous 
while all others (except sex) are con- 
tinuous, a situation which tends to hold 
down the size of the coefficients* 

Table 4.5 presents the summarized results of the multiple 
correlation technique described previously. The first column 



o 
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Table 4,5 

The coefficient of multiple correlation for each diagnostic 
group under three conditions. (1) For each group individually 
based on own data, (2) for each group based on the weights for 
the schizophrenic primary group, (5) for each group based on 
the weights for the combined diagnoses. 





(i) 


Group 


Own Data 


Schizophrenic 

Primary 


-4799 


Schizophrenic 

Criterion 


.5475 


Schizophrenic 

Combined 




Manic Depressive 


.4817 


Chronic Brain 
Syndrome 


.4461 


Personality 

Disorder 


.5940 


Mentally 

Deficient 


.3085 


Total Group 


.5490 



(2) 


(3) 


Schizophrenic 

Primary 

Data 


Whole Group 
Data 


.4799 


.4653 


.5248 


> 5269 




.4942 


.3999 


.4171 


.3119 


.3782 


.5062 


.5695 


.1810 


.1714 


.5392 


.5490 
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presents the correlation between predicted and actual outcome 
when the predictions were obtained from score weights com- 
puted for each group separately. The weights used are pre- 
sented in the statistical supplement. All correlation co- 
efficients are significantly different from zero except that 

for the mentally deficient group. 

The second column shows the coefficients obtained when 
the weights found for the schizophrenic primary group are 
applied to the other groups. There was a substantial decline 
in the size of the coefficients except in the schizophrenic 
criterion group where the decline was negligible. These 
weights do well, as might be expected, for the schizophrenics 
but are markedly less successful when applied to the other 
diagnostic groups. It seems safe to conclude that the weights 
best for the schizophrenic group cannot be applied to the 
other diagnoses. 

A third approach to the same problem was made by using 
the weights generated for the total group. The results are 
presented in Column J. In most cases there is an increase in 
the size of the coefficient ever that obtained when using the 
schizophrenic primary weights. One schizophrenic group ex- 
hibits a small decline, an observation which is quite pre- 
dictable since the weights in the second column are based 
entirely on the schizophrenic segment of the group. 

The table may best be summarized by saying that the reader 
must make his own decision whether to use the individual diag- 
nostic group weights or the whole group weights. It is the 
bias of the writer that the increase in the coefficients ob- 
tained by using individual diagnostic group weights is of 
negligible value. The best measure of the difference is to 
square the coefficients to obtain the per cent of the variance 
accounted for by the correlation. This indicates, when using 
the whole group weights, an average decrement of four per cent 
with the maximum of seven per cent occurring in the mentally 
deficient group. 
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At this point in the analysis we are faced with the 
practical problem, to wit, is the technique possessed of 
sufficient accuracy to "be practically worthwhile. The most 
satisfactory approach to this problem is to find out how well 
the technique can do at making individual predictions and then 
simply counting up successes and failures. 

It will be remembered that the criterion variable was 
dichotomous, being coded as (1) for 'in T and (2) for 'out'. 

When the computer applied the weights to the obtained scores 
for each individual, it obtained a numerical value which was 
the predicted outcome for that person. These obtained values 
ranged from about .75 to 2.10. Values less than 1.501 were 
considered to be predicting an 'in 1 and values at or above that 
point to be making an 'out' prediction. The predicted outcome 
could then be compared to the actual outcome and the results 
tabulated in a four-fold table. Two categories, those where 
the predicted and actual outcomes agreed, can be counted as 
successes and the remaining two categories as failures. The 
results of this analysis are presented in Table 4.6. For ea.se 
in comparison, the results obtained by using predictions based 
on 'own-group' and 'whole-group* weights are shown in the same 
table. 

There are several points of interest in Table 4.6. 
Primarily we are interested in the accuracy with which the 
tests can classify the individual patient. Inspection of the 
'per cent correct* columns indicates that the tests do a highly 
satisfactory job of prediction. The values for all groups are 
sufficiently alike to make it clear that no one group accounts 
for a large share of the accuracy. While this point was not 
specifically tested, it seems clear that the between-group 
differences are probably not of practical importance. 

Comparison of the two methods indicates a small difference 
in favor of using the weights obtained for each diagnostic 
group. However, it again seems clear that this difference is 
of no practical importance. Inspection of the totals indicates 
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that 22 more patients are classified wrongly under the whole- 
group weights than under the individual-group method# A two 
per cent increase in accuracy is probably not worth the com- 
puter time involved. 

At this point it may be well to pause for discussion of 
certain aspects. It takes no great amount of imagination to 
sense a certain amount of reader credulity about now. The per 
cent of correct classification or prediction seems a bit too 
good to be true. A critical check for possible sources of 

error is certainly indicated. 

First, let us examine the data for internal consistency o 
The analysis of variance (Table 4.2) indicated a significant 
difference betureen the ‘in* and 'out' groups in performance 
on ail tests for all except the mentally deficient category. 
Turning to the coefficients of multiple correlation as reported 
in Table 4.5, the square of the whole-group coefficient, .5490, 
indicates that this correlation accounts for only 30 per cenu 
of the total variance. This result appears at odds with the 
apparent accuracy of prediction reported above. 

A bit of reflection provides a rational explanation. It 
will be remembered that we are predicting a dichotomous 
variable, 'in’ or ’out 1 from a set of essentially continuous 
variables (test scores, age). But in order to make the pre- 
diction we were forced to dichotomize the continuous prediction 
variable about the arbitrary value of 1.500. Hence our per 
cent correct classification system ignores the variation within 
the groups thus created. The two sets of results thus are 
really not inimicable. 

There is another source of error which must be considered. 
It is possible to obtain a correlation between two sets of 
data which is due largely to their individual relation to a 
third variable. Analysis of the methodology used in this study 
does not support any logical third-factor hypothesis. There 
was complete independence between the test scores and outcome 
criterion in the sense that the scores could not influence the 

outcome. 
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It has been suggested that a third factor is psychiatric 
status. This is plausible since it has been demonstrated 
(King 1954 » Brooks and Weaver 1961) that psychomotor performance 
is positively related to psychiatric status and that the latter 
is obviously related to hospital discharge. However, the 
argument is quite circular since the * in-out 1 criterion was 
chosen as the most reliable and objective measure of psychiatric 
status which was available on so large a group of patients. 

We believe that the measures and criterion were independent in 
fact and that the obtained relationships do not result for a 
fortuitious combination of circumstances. 
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Chapter 5 



PSYCHOMOTOR TESTS RELATED TO PATIENT OUTCOME: 

MULTIPLE CUTTING POINT METHOD 

The multiple cutting point procedure is also borrowed 
from industrial psychology. In this approach the test user 
usually has a formulated objective in the sense that he may 
wish to include or exclude a certain proportion of the group, 
or he may wish to maximize successful selections while mini- 
mizing unsuccessful selections. The point is that the cutting 
point selected must reflect the purpose of the user. Hence a 
wide variety of cutting points may be applied to the same score 

distribution# 

In thic chapter we shall use the same schizophrenic pri- 
mary and criterion sub-groups as we described in the previous 
chapter. The data from the primary group will be used to 
develop a set of cutting points. These points will then be 
validated by applying them to the criterion group. 

The analysis was begun by constructing score distributions 
for each group on each of the five tests. These distributions 
listed the frequency at each score interval for the 'in' and 
•out' groups separately. A variety of approaches to selecting 
the "best" cut-off point were tried and discarded. An 
apparently promising method involved grouping the distributions 
into appropriate intervals, converting the N's for the 'in' 
and 'out' groups to per cent frequency, and then computing the 
difference in per cent on each interval. Presumably the in- 
terval with the greatest difference would be the location of 
the cutting point. The difficulty arose from the fact that 
the interval with the greatest difference is usually so far 
down the distribution than an unduly large portion of the 'm' 

group is above tb© cut-off point* 

Reflection on the probable use of the test battery would 

appear to indicate that it would be most useful in arraying a 
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group of potential rehabilitation candidates in terms of 
probability of release. The most usual danger to be avoided 
in these circumstances is the inclusion of too many potential 
•ins' whose presence would tend to cut down on the productive- 
ness of the program. Therefore, we have adopted, as a working 
hypothesis, a cut-off point which would include 67 per cent of 
the 'out' group. For convenience in exposition, cut-off points 
of 35 per cent and 50 per cent are included in the same table. 

Table 5*1 represents a summary of the general outcome of 
this procedure. The table was constructed in the following 
manner. For each test, the score points which included 1/3? 
1/2, and 2/3 of the 'out* sub-group of the primary group were 
determined. These same cut-off points we re then applied to 
the data for the primary 'in' sub-group and to the criterion 
'in' and 'out 1 sub-groups. 

Several comparisons should be made in Table 5*1* First 
is the obvious difference between the primary ’out* and 'in* 
per cent at all levels of the five tests. The cut-off point 
of primary interest to us (67 per cent of the ‘out 1 group) 
regularly includes about 1/3 of the ■in 1 group. We may con- 
clude that there is sufficient agreement among the tests to 
make us confident of the reality of this result. 

The next and crucial comparison involves the outcome of 
application of the cutting point to the independent criterion 
group. These data are reported in the two right-hand columns 
of Table 5.1 as per cent of the group above the cutting point. 
Within this group, comparison of the ^n* and 'out* sub-groups* 
shows a highly satisfactory degree of separation, in fact, 
slightly better than was achieved with the primary sub-groups. 

The degree of agreement between the primary and criterion 
groups is sufficient to warrant combining them. The results 
of the combination are reported in Table 5*2 which also con- 
tains an overall summary figure obtained by averaging across 
tests. This final figure is remarkably close to the desired 
cut-off proportions. It seems quite evident that the cut-off 
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The effects of using cut-off points for 
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Averages 




50 


23 


50 


18 






67 


36 


69 


31 




Table 5.2 



The per cent of the schizophrenic 'out* and 'in 
groups lying above the three cut-off points 
for each of the five tests. 
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points thus generated do an adequate job differentiating 
the schizophrenic sub-groups# Obviously, other sets of cutting 
scores can be generated for specific purposes. The current 
ones appear to have some generality of application and serve 
as well for illustrative purposes. 

The question of whether these standards developed on 
schizophrenics can be applied to other diagnostic categories 
as well is partially answered in Table 5*3 which shows the 
per cent of each diagnostic group scoring above the three cut- 
off points on each of the five tests. It must be remembered 
that the cut-offs were designed to yield about 1/3 > 1/2, 2/3 
of the schizophrenic group. In a real sense, the values of 
the schizophrenic 'out* group are the independent variable 
and all other values are dependent variables. 

It is obvious by inspection that there is considerable 
grou.p x test interaction. Certainly there is nothing like 
uniformity of selections within the various diagnostic cate- 
gories. Nor is there any real reason why we should expect 
to find such uniformity. The analysis of variance previously 
showed that there were numerous significant between-groups 
differences with the chronic brain syndrome and mentally 
deficient groups usually being significantly poorer in 
perf ormance# Hence a uniform cut-off score could hardly be 
expected to yield uniform proportions of the differing groups. 

The data appears to support the following conclusions 
and implications, 

(1) The cut-off points generated on the 
schizophrenic primary group did differ- 
entiate between the ’out’ and ’in' 
outcome groups. 

(2) These same cut-off points when applied 
to the schizophrenic criterion group 
were successful to about the same extent 
in differentiating between the 'out' 
and ’in’ outcome groups. 
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(3) Application of the cut-off points to 
patients of the other four diagnoses 
were less successful. This appears 
to he due in some measure to the 
variations in discharge rate within 
these diagnostic groups. 
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Chapter 6 

CLINICAL APPLICATION OP TEST DATA 

This chapter will he concerned with the specific 
applications of the test data to the individual patient, 
illustrating the manner in which the test data can he applied 
and interpreted. There are two basic approaches to the 
problem: interpretation through decile scores, and prediction 

of outcome through use of a weighted combination of test scores. 

We will consider first the use of decile scores since it 
is the most straightforward and easiest to understand. The 
basic output of the test program is a set of scores for each 
patient. For our purposes it is immaterial whether these are 
the result of a single day of testing or the mean score across 
five days. All that is required is a set of numbers represent- 
ing performance on each of the five tests. 

The basic reference data are contained in Tables 6.1 
through 6.5 in the columns headed ’Decile Limits’. These data 
will enable the user to place the score in the appropriate 
decile. The result is five decile scores. These may be 
averaged and considered as a single score or they may be con- 
sidered individually if desired. If there is a wide scatter 
to the decile scores, it is probably inadvisable to average 
them. 

The scores may be interpreted in a variety of ways. The 
decile furnishes information about the placement of the in- 
dividual with regard to the performance of the whole group. 

A decile score of VIII indicates that the performance exceeds 
or equals that of 80 per cent of the group of approximately 
1000 patients. 

The tables can also be used to obtain an approximation 
to the probability of discharge. The tabular entries are in 
per cent cumulated from top to bottom. Either the columns for 
the appropriate diagnosis or for the whole group may be used, 
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at the option of the user. In this connection it should he 
remembered that the diagnostic groups with smaller number of 
cases may show a relatively greater fluctuation in the per 
cent figures as the result of a small difference in numbers. 

In these cases, consideration of the whole-group data as well 
as the diagnostic-group data is recommended. 

Let us consider an example as a means of illustrating 
ho v/ the tables may be used. Assume a score of 30.00 on the 
transport test for a patient diagnosed as personality dis- 
order*. From Table 6.3? si score of 30.00 falls in Decile VII. 
For the whole-group data, the cumulative per cents for the 
*in* and *out* groups are 21 per cent and 58 per cent re- 
spectively. For the 1 personality disorder* diagnosis, the 
same values are 24 per cent and 68 per cent. Since there is 
little difference bet ween the two sets of values, the same 
interpretation would hold for both. There are about two 
chances in ten that the individual would fall in the ’in* 
group and about six in ten of falling in the 'out* group. 
Since the score itself falls near the upper limit of Decile 
VII, the above probabilities are somewhat conservative. 

To further elaborate the process, we will examine in 



detail the data obtained on one of our cases. This randomly 



chosen example was a male schisophrenic, 20 years old at test- 
ing, who was out of the hospital by the end of the obser- 
vation period* For convenience, the data will be arranged in 

tabular form. 



Test 

Tapping 

Reaction Time 

Transport 

Assembly 

Serial Reac- 
tion Time 



Score 


Decile 


21.40 


VII 


23.80 


VI 


25.40 


IX 


42.27 


VI 


13.09 


VII 




VII 



Schisophrenic 



Out 


In 


5/10 


2/10 


6/10 


3/10 


3/10 


1/10 


7/10 


3/10 


5/10 


2/10 


5/10 


2/10 



Whole 


Group 


Out 


la 


6/10 


2/10 


7/10 


2/10 


3/10 


1/10 


7/10 


2/10 


6/10 


2/10 


6/10 


2/10 



3 

ERIC 



Average 



These figures may be evaluated by inspection. For the 
schizophrenic data, there are about five or six chances in ten 
that the individual is in the 'out* group and two or three 
chances in ten of being in the 'in' group. Since these 
’probability' figures are computed for each group separately, 
they need not account for ten chances in ten. Essentially the 
same comments apply to the ’whole group' data. 

Care must be taken in the interpretation of these 
probability figures. In reality, they are based on the fact 
that the 'out' group is much more heavily represented in the 
upper deciles than the 'in' group. As the middle and lower 
deciles are approached, the discrimination becomes much poorer. 
The prediction value is largely based on the difference between 
the cumulative per cent figures for each decile. While this 
forms an obvious limitation on the use of the tests, the 
practical difference is of considerably lesser importance „ 
Generally speaking* we are interested in selecting those 
patients best able to profit from training* For the most part, 
these will be found in deciles VI - X where the discrimination 
is relatively satisfactory. 

In short, we feel that this approach offers a useful tool 
to the clinician which will permit him to make an intuitive 
estimate of the patient’s performance with reference to two 
sets of standards. It is not offered as a research-quality 
technique which will permit a definite prediction for each 
patient. 

There is another approach which will permit a definite 
though not necessarily more accurate — prediction for each 
patient. It will be remembered from the material presented 
in Chapter 4 on multiple regression and multiple correlation 
that the process was based on selecting the best combination 
of weighted test scores. The output of the process was a 
number ranging from slightly less than 1 to slightly more than 
2. These predictions were interpreted as indicating an 'in' 
prediction when falling at or below 1.50 and as an 'out' for 
all values above that point. 
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The equation from which the prediction is made consists 
of a constant to which is added algebraically the product of 
the test scores multiplied by the calculated b— weights. The 
values for the various constants and b-weights for the various 
groups and test combinations will be found in the appendix. 

To illustrate the method , we shall use the set of weights 
obtained when the five psychomotor test scores are used as 
predictors. The test scores used are those which were used 
in the discussion of the decile score technique. The general 
form of the equation is given below. 

Prediction = A (constant) + B 1 (tapping) + B 2 (reaction 

time) + Bj (transport) + B^ (assembly) + 

B c (serial reaction time) 

Inspection of the values from the appendix makes it clear 
that the numbers are not adapted for easy handling. Due to 
the necessity for making the predicted value not exceed a 
value much greater than 2.0, the b— weights have significant 
values in the sixth decimal place. To further complicate 
matters, the computer program for this analysis treated all 
score values as if they were whole numbers. Consequently, in 
the computation of the prediction, decimals in the score values 
must be omitted. As will be seen in the following example, 
it is most convenient to move the decimal point in the 
constant and b— weights six places to the right before begin- 
ning computations and then, when the arithmetical operations 
are finished, to move it bafek six places to the left. We 
will illustrate the computation by both methods. It should 
be noted that the b-weights have algebraic signs which must 
be taken into account in the computations. 
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Test 


Score 


fc-weigh-t 


+ 




Tapping ' — 


3140 


_ .000078 


. 244920 




Reaction Time 


2380 


-.000045 




. 107100 


Transport 


2540 


. 000016 


. 040640 




Assembly 


4227 


- . 000089 




. 376203 


Serial Reaction Time 
Totals 


1309 


-.000023 


. 285560 


. 030107 
. 513410 



1.860000 + .285560 - .513410 » 1.652150 (Prediction) 



In this example, the obtained value exceeds 1,500 and thus 
(correctly) predicts an * out 1 . If it is desired to avoid the 
rather cumbersome decimal calculations , we can move the 
decimal point six places to the right in the A value and 
b-weights. This results, in the instance above, in the dis- 
appearance of the decimals in all the computations. In the 
final value, the point would follow the terminal zero. As the 
last operation the point is moved six places to the left and 
the value obtained in the original solution reappears. 

So long as the operator takes due care with the signs and 
decimal points, the operation is quite simple and straight- 
forward. Use of a desk calculator permits direct solution of 
the equation through positive and negative multiplication 
operations. 

The question of which system is better or more useful 
is one which must depend upon the user for determination. 

Both methods will produce valid and useful information. 
However, it cannot be emphasized too often that these tests 
measure only one aspect of an essentially complex situation. 
The final judgment must rest on the shoulders of the skilled 
clinician. 

We wish to enter a precautionary word about confusing 
mathematics with pi'ecision. The fact that the b~weight 
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technique may give rise to an answer mathematically correct 
to the sixth decimal place does not imply a validity of equal 
precision. This means that the predicted value should not , 
when other factors are judged favorable, be allowed to function 
as an absolute bar to selection. This is particularly true of 
the mid-range values near the cut-off point of 1.500* While 
we have not been able to develop a satisfactory measure oi 
reliability for our prediction number, it is highly probable 
that there is mo significant difference between a value of 
1.49 and 1.51. We recommend that in all borderline cases the 
patient be given the benefit of the doubt. 
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Chapter VII 

DISCUSSIONS AND IMPLICATIONS 

The implications of the foregoing tody of data will' he 
discussed under two aspects , practical and theoretical. The 
major portion of the discussion will he devoted to the first 
aspect. 

In our view, these tests have two primary applications. 

One is to enable an administrator to arrange a group of 
patients in the order of readiness for rehabilitation training, 
a type of problem frequently encountered in organizing or ex- 
panding a rehabilitation program. The other application is 
to enable a clinician to evaluate the performance of a patient 
either with respect to local performance norms or with respect 
to the norms contained in this report. 

The first application mentioned, that of group ordering, 
is probably the most important use of these tests. It is im- 
portant that the selection of patients for rehabilitation 
training, particularly in a mental hospital where the supply 
of candidates may far outstrip the number of available program 
spaces , be conducted so as to select patients with a high 
probability of success and avoid the slower-moving patient who 
may require a much longer time to get through the program. 
Depending upon the relation between supply and capacity, the 
selection system may be as strict or as lenient as the user 
deems advisable. 

Candidates for rehabilitation could be selected so as to 
insure the early and continuing success of a program, thus 
maximizing the morale of both the patients and the rehabili- 
tation team. They could be used to divide a population of 
patients into groups which would need brief training and which 
would need more prolonged programs. They could also be used 
to eliminate the least hopeful cases and those for whom little 
or no rehabilitation effort would be needed. 
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Through differential consideration of the individual test 
scores, they should also prove very useful in selecting 
patients who would he capable of work requiring fine finger 
dexterity and those who should be trained for tasks requiring 
only more gross movements. 

A word of caution and clarification should be inserted 
into the discussion at this point. It must be emphasized that 
these tests are an adjunct to and not a substitute for clinical 
judgment. They cannot be used for diagnostic purposes. Despite 
the significant differences among the diagnostic groups which 
have been demonstrated, the tests are completely unable to 
distinguish, for example, between a phobic personality dis- 
order and a potentially dangerous paranoid schizophrenic. 
Indeed, it is quite probable that both would achieve scores 
which would indicate high potential for rehabilitation. The 
differentiation of these conditions is the task of the clini- . 
cian and not that of the psychomotor test technician. 

What these tests can do in the clinical situations is to 
suggest an order of consideration. If the program of testing 
is hospital-wide so that data are available on all patients, 
the population may be screened for high-scoring patients who 
have not been considered for inclusion in the rehabilitation 
program. It is not unusual for the quiet, well-behaved patient 
to be overlooked. In the course of this study we located 
about half-a-dozen such patients who were then recommended 
for consideration. Almost all of this group have subsequently 
left the hospital. We feel that the degree of contamination 
of our data, less than one per cent, could be accepted in view 
of the other benefits involved. 

Also pertinent in this regard is the question of the 
extensiveness of a testing program. It is recommended that 
a hospital-wide program be given serious considerations if a 
rehabilitation effort of some magnitude is contemplated. 

This approach not only furnishes the maximum amount of infor- 
mation but also facilitates the development of a local set of 
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norms on the tests* While we have reason to doubt that these 
norms would differ materially from those presented herewith, 
their presence would lend confidence to the interpretation of 
performance data* However, when a program involving only a 
small number of patients relative to the hospital population 
is to be undertaken, the more extensive program would probably 
not be justified. It would be desirable, however, to test all 
patients who were considered eligible for training in order 
to obtain as broad a sample as possible from which to select. 

The second area of application ns that of the evaluation 
of the individual patient where large-scale testing is not to 
be accomplished. This situation provides several limitations 
on the use of the tests. The user must depend, at least 
initially, on the norms published herewith. This implies that 
the tests to be used are substantially the same as those used 
in this study if validity is to be preserved. Elsewhere in 
this report are detailed specifications of the physical size 
and arrangement of the test apparatus. We cannot specify that 
some or any of these details and specifications are critical 
but the user who departs from them materially does so at an 
unknown degree of risk. 

Assuming a substantial replication of the test situation 
and procedures, the user can compute the mean performance on 
each test and refer it to the decile limits as contained in 
Tables 6.1 to 6.5. Based on this information, he can then 
place the patient with reference to the appropriate group and 
determine the per cent of people with similar scores who left 
or remained in the hospital. From this poinvr on the clinician 
is on his own in solving the complex equation of factors pre- 
sented by the individual patient. We do not feel that a low 
score should always serve as an absolute bar to rehabilitation 
training but the clinician must be prepared to accept the 
probable consequences of such a selection. Each user must 
assay his local situation and make his decision accordingly. 

In t hi s connection, it is important to take cognizance 
of the currency of the test data. We have previously remarked 
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that we feel the test performance is strongly related to 
psychiatric status. It is not unusual for this status to 
change (else there would he little point to therapeutic 
effort), sometimes with considerable celerity. Where there 
is doubt as to whether the test results accurately reflect 
the current status of the patient, a re-test is certainly in 
order. Re-testing at intervals should help tc indicate which 
patients are responding to adequate treatment and which need 
some change in management. The tests may also be of assist- 
ance in making the sometimes painful .decision to terminate 
the rehabilitation program of a patient as not presently 
feasible. 

The theoretical implications of these data are markedly 
less obvious than the practical ones. There are, nonetheless, 
certain elements which seem to fit together and which appear 
to have certain theoretical aspects. We shall briefly restate 
certain points to insure a common ground for discussion. 

In the previously cited Columbia-G-reystone project 
report (1949), King observed that, following prefrontal 
lobotomy, the psychomotor test items showed improvement in 
performance before there was any evidence of clinical im- 
provement. The post-operative period was marked by a con- 
siderable decrement in all forms of behavior and the psycho- 
motor test items provided the first index of recovery. From 
this we infer a relationship between psychomotor performance 
and the functional state of the brain during its recovery from 

cerebral insult. 

The second point is the monograph by King (1954) which 
examines the experimental evidence regarding the relationship 
between psychomotor impairment and severity of mental disease. 
He demonstrated clearly that there was a direct relationship 
between amount of impairment and rated severity of mental 
disease. Work in our own laboratory (Weaver 1961) confirmed 
King*s findings to a degree limited only by the commonality 
of the two experimental situations. On the basis of these 
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results, we cannot exclude the possibility that the mental 
disease process is due to interference with normal cerebral 
function by some unspecifiable condition or agent. 

The third point is the demonstration (Brooks and Weaver 
1961) that substitution of placebo for active tranquil! zing 
medication results in decrement in both' the psychomotor test 
performance and psychiatric status of the patient. In both 
the deterioration and the recovery following restoration of 
tranquilizing medication phases , the changes in test per- 
formance preceded the clinically observable behavior changes. 
Since it seems generally agreed that tranquilizing medication 
has a central rather than a peripheral locus of agent, we 
again infer that the psychomotor test results reflect the 

state of the cerebral function. 

The implication which we draw from these lines of evidence 
and from the present study is that we can conceive of mental 
disease as a process usually characterized by three phases; 

(l) onset, with concurrent decrement in the adequacy of be- 
havior, (2) illness, with continued impairment of function, and 
(3) recovery, during which there is improvement in function. 
This improvement may be spontaneous or influenced by tran- 
quilizing medication. 

The mode of action of the psychomotor test would thus 
appear to be the measurement of the central state of the 
nervous system. Since the clinically observable behavioral 
change appears to lag substantially behind the changes in the 
psychomotor test performance, the result is that the psycho- 
motor test results tend to predict the behavioral changes. 

This theoretical account of the method by which the 
psychomotor tests 'operate is advanced as the most straight- 
forward account which we can give in the process as we under- 
stand it. It does not attempt to shed new light on the 
schizophrenic process (whatever that may be) other than to 
insist that the process has its locus of actxon in the braxn 
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and that such modifications of the process as can presently 
he effected are accomplished in the same organ. 

Whether or not this explanation is accepted is really 
not germane to the issue of this report. By whatever method 
it is accomplished, the psychomotor tests do differentiate 
substantially between those who get out of the hospital and 
those who remain. CPhe application of this information need 
not wait upon a universally accepted theoretical account of 
why things are as they are# 
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Table 


A-l 








Intercorrelations among 


the various predictors 






and outcome for the total 


group. 








Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 


Sex 


101 


-.091 


.119 


-.040 


-.009 


.020 


.109 


Age 




-.232 


.362 


.208 


.305 


.270 


-.188 


Tapping 






-.632 


-.518 


-.622 


-.604 


.441 


Reaction 

Time 








.496 


.576 


.645 


-.436 


Transport 




' 






,847 


.635 


-.409 


Assembly 












.619 


-.493 


Serial 

Reaction 

Time 


• 












-.412 








Table A-2 








Intercorrelations among 


the various predictors 




and 


outcome for 


the schizophrenic primary 


group. 






Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 


Sex 


.077 


-.090 


-.031 


-.056 


.018 


.000 


.102 


Age 




-.233 


.319 


.240 


.317 


.287 


-.249 


i Tapping 






-.629 


-.408 


-.521 


-.597 


. .327 1 


Reaction 
, Time 


• 






.529 


.576 


.719 


-.403 


1 Transport 










.788 


.534 


-.345 


Assembly 












.496 


-.391 


Serial 

| * Reaction 

Time 

# 

m 














-.345 ; 


i . 

ERIC- 






























Table A-3 



Intercorrelations among the various predictors 
and outcome for the schizophrenic criterion group. 





Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 


Sex 


.152 


-.158 


.017 


.062 


.096 


.118 


.062 


Age 




-.238 


.384 


.210 


.350 


.281 


-.24-3 


Tapping 






-.622 


-.436 


-.373 


-.342 


.402 


Reaction 

Time 








.418 


.543 


.597 


-.503 


Transport 










.780 


.457 


-.281 


Assembly 












.467 


-.399 


Serial 

Reaction 

Time 














-.386 



Table A-4 



Intercorrelations among the various predictors 





and outcome 


for the 


manic depressive 


group • 






Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 


Sex 


-.089 


-.234 


.195 


.043 


.062 

1 


-.079 


-.045 


Age 




-.145 


.299 


. 304 


.324 


.298 


-.179 


Tapping 






-.537 


-.565 


-.654 


-.529 


.352 


Reaction 

Time 








.601 


.629 


• 582 


-.367 


Transport 










.901 


.624 


-.307 


Assembly 












.664 


-.430 


Serial 

Reaction 

Time 














-.322 
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Table 


! A-5 










Intercorrelations among 


the various predictors 






and 


outcome for 


the chronic brain syndrome 


group • 








Age 


lap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 




Sex 


.170 


-.124 


.208 


.078 


.120 


.095 


.092 




Age 




— . 122 


.275 


.113 


.183 


.179 


.131 




Tapping 






-.594 


-.612 


-.658 


-.522 


.322 




Reaction 

Time 








.609 


.520 


.656 


— • 211 




Transport 










.860 


.732 


-.294 




Assembly 












.639 


-.353 




Serial 

Reaction 

Time 










• 




-.242 










Table A-6 










Intercorrelations among 


the various predictors 






and outcome for the personality disorder group. 








Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 

React. 

Time 


In/Out 




Sex - 


.027 


-.197 


.073 


.082 


.078 


.017 


.054 


\ * 


Age 




.008 


.328 


.130 


.153 


.289 


-.078 




Tapping 






-.528 


-.554 


-.566 


_ cr/iQ 
— * ^tO 


.426 




Reaction 

Time 








,415 


.466 


. 411 


-.330 




Transport 










.884 


.640 


.432 




Assembly 












.659 


-.532 




Serial 

Reaction 

Time 














-.563 i 
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Table A-7 



Intercorrelations among the various predictors 
and outcome for the mentally deficient group. 





Age 


Tap- 

ping 


Reac- 

tion 

Time 


Trans- 

port 


Assem- 

bly 


Serial 
React e 
Time 


In/Out 


Sex 


» 

.094 


-.241 


.106 


.080 


.047 


.193 


-.106 


Age 




-; i99 


.497 


.520 


.492 


• 226 


-.265 


Tapping 






-.449 


-.490 


-.515 


-.577 


.111 


Reaction 

Time 








.448 


.573 


.413 


-.199 


Transport 










.750 


.606 


-.063 


Assembly 












.582 


-.141 


Serial 

Reaction 

Time 














-.114 
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Decile limits and distribution of cases by diagnosis and outcome 
for the transport test. Entries are number of cases. 
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gure A-l The mean performance on the tapping test on each 
day for the five diagnostic groups. 
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Figure A*3 The mean performance on the transport test on 
each day for the five diagnostic groups. 




A very simple and uncomplicated demonstration of the 
selection power of the tests was made by using the cut-off 
scores mentioned in Chapter 3« For each subject, the mean 
score on each test was compared with the cut-off score (point 
which cut off two-thirds of the ’out* group). Performance 
exceeding the cut-off was defined as 'passed', below as 
'failed'. Thus each individual could be placed in one of 
six classes ranging from '0 passed' to '5 passed*. The results 
of this analysis arranged by diagnostic group and outcome are 
presented in Table A-19 and graphically in Figures A-6 through 
A-ll. 

Before considering these data in detail, it would be well 
to review certain aspects previously mentioned. The schizo- 
phrenic group was selected for analysis not only because it 
was the largest, but also because it had the in/out ratio at 
nearly 50 per cent; 43/57 to be exact. In at least two of the 
diagnostic groups, this ratio is badly out of balance, being 
9/1 in the mentally deficient group and 8/2 in the chronic 
brain syndrome group. This factor will obviously place severe 
limitations on the selection ability of the tests. In the 
instances where this imbalance results in small N's, it may 
well be necessary to ignore the statistically unreliable be- 
havior of these small groups and consider only the more stable 
larger subgroups. 

The phenomena under discussion can be most readily 
visualized from the graphs. Figure A-6 for the schizophrenic 
group shows a typical relationship between the 'in' and 'out' 
groups when there is a positive relationship between outcome 
and number of tests passed. The two curves are nearly 
symmetrical with no reliable difference only at point 2 and 3 * 

Considering next Figure A-7 for the manic depressive group, 
the curve for the 'out' group, which is based on 70 cases, is 
quite similar to that for the 'out* schizophrenic group. The 
curve for the 'in' group, with 14 cases distributed on six 
intervals, is essentially random. 
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The came effect in reverse is seen in Figxire A-8 for nhc 
chronic brain syndrome group. Here the curve for the 'in* 
group* hased on 81 cases, is stable and in general conformity 
to the similar curve for the schizophrenic group. 

Essentially similar to the curve for the manic depressive 
'out' group is the curve for the personality disorder group 
in Figure A-9, although the number of cases passing all five 
tests is reliably greater for this group. The 'in' group is 
essentially random. 

Possibly the best example of the effects of a small N is 
contained in Figure A-10 which shows the data for the mentally 
deficient group. With a total of eight cases in the 'out', 
the three which fall into the 'one passed* category boost the 
per cent way up to 37*5 P©** cent. The form of the 'in' group 
curve is quite usual except for the pile-up in the 'none passed 

category. 

Figure A-ll shows the result of an attempt to display the 
results of the selection technique when the imbalance previously 
discussed is minimized. This was accomplished by creating a 
composite curve from which x*ere eliminated those small elements 
discussed above which tend to upset the per cent base unduly. 
Those elements eliminated were: manic depressive 'in', chronic 

brain syndrome 'out', personality disorder 'in', mentally 
deficient 'out'. The result is a fairly regular and symmetric 
curve which clearly portrays the difference in perfoxmance of 
the 'in' and 'out' groups. 
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Figure A-7 The number of tests passed for the 'In' and 'Out' groups 
of the manic depressive diagnostic category. 
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figure A~13 Circuitry for selector switches, discs, and light for serial rsaction time test 

Lettered points connect to matching letters on other diagrams 
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figure A -15 Circuitry for tapping and reaction time test 
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figure A - 17 Capacitance operated relay for transport assembly test 
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